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1. Introduction 

A system contains design flaws, each of which eventually manifests 
itself at some point in time, whereupon the system is redesigned in order 
to remove the design flaw. Design flaws are often called "bugs," and the 
time points mentioned above will be called "failure times." If the 
failure times are indexed chronologically they can be represented as: 

0 < s^ < s^ < < ... . (1.1) 

It is convenient to consider (1.1) as the realization of a random process: 

0 < S 1 < s 2 < s 3 < ••• , ( 1 . 2 ) 

where the S. ’ s are random variables and the s.’s are real scalars, 
i 1 

If system redesigns successfully remove design flaws, system reliability 



will improve and (1.1) and (1.2) should show a general pattern of increasing 
interfailure times, i.e., reliability growth. This paper presents and 
investigates a family of probability models for the failure-time process 
in (1.2). They are reliability growth models called Exponential Order 
Statistic models. 

Exponential Order Statistic (EOS) models are based on the following 
assumptions: Each design flaw has a rate of occurrence; a flaw causes 

failures to occur according to a Poisson process if it is not removed; 
and design flaws cause failures independently of one another. If each flaw 
is removed the first time it causes a failure, the overall failure- time 
process corresponds to order statistics from independent but not necessarily 
identically distributed Exponential random variables. Campbell and Ott [4] 
use such a model. The model is implicit in Nagel’s [21,22] replicated 
software debugging experiments. Adams [1] studies the rates at which 
design flaws cause failures. 

Three different subfamilies of EOS models are considered. These 
are based on three classes of models for the failure rates of the design 
flaws initially present in the system: (i) a collection of deterministic 

rates, (ii) a finite set of random independent identically distributed 
rates, (iii) a set of random rates whose joint distribution is that of a 
nonhoraogeneous Poisson Process (NHPP) . Models of type (i) are deterministic 
EOS models (denoted DET/EOS) , models of types (ii) and (iii) are doubly 
stochastic EOS models (denoted DS/EOS) . Various properties of and inter- 
relationships between these three families are given. 

The ultimate goal when developing reliability growth models is the 
development of good reliability inference and prediction methods which 
are applied to software development and maintenance. Nevertheless, 
this paper will not deal with inference or prediction per se. The focus 
is on the probability models and their properties. However, some of these 
properties do provide bases for new approaches to estimation and prediction 
of reliability growth. 

The general content of the paper is as follows: Section 2 contains 

definitions, basic properties and some examples of EOS models with 



deterministic failure rates. Section 3 introduces doubly-stochastic EOS 
models and tries to portray them as natural extensions of the DET/EOS 
models. Section 4 presents some general mathematical results as well as 
some guidelines dealing with problems in distinguishing different models 
from each other. Section 5 presents an important parametric family of 
DS/EOS models: the Gamma/EOS models; the closure of this family includes, 

as special cases, the Jelinsky-Moranda [12] model, the Goel-Okumoto [10] 
model, the Littlewood [16] model, the Musa-Okumoto [20] Logarithmic 
Poisson model, and the Power Law Model [6,7]. In Section 6 the mean 
function and intensity of the failure-time process (1.2) are characterized 
using the property of complete monotonicity; this will be useful for 
nonparametric analyses. Section 7 includes more parametric families of 
EOS models and additional properties. Section 8 contains some brief 
comments on possible approaches to inference and prediction using the 
results of this paper. Conclusions are in Section 9. An Appendix 
contains proofs and other details omitted from the body of the paper. 

2. Exponential Order Statistic Models 

Let X. , i=l,2,...,n, be independent Exponential random 
variables with rates A.^ , i=l,2,...,n , respectively. (Throughout this 
paper we assume 0 < A. < » .) if l [0>t] (O is the indicator function 
of the set (x: 0 < x < t} , then 

n 

N(t) = i-1 > 0 . (2.1) 

equals the number of events occurring by time t . Let S. = mintt: N(t) > i}, 

i= l»2,...,n , be the occurrence time of the ith event, i.e., the ith order 

statistic of , . . . ,X^ . The stochastic processes (N(t), t > 0} 

and { S^ , i=l, 2. . . } are the counting process and the occurrence- time 

process, respectively, of an Exponential Order Statistic (EOS) model with 
parameter set A = { A^ , A^ , ...,A } . Anticipating the introduction of 
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doubly stochastic models in the next section, we shall denote the above 
model as a deterministic EOS model, abbreviated as DET/EOS. 

It is mathematically possible and convenient to permit an infinite 
number of events, i.e., n = 00 . In this case the parameter set X is 

O, 

00 

infinite. If T X. < 00 , then P(N(t) < °°) = 1 for 0 < t < 00 , 
i=l 1 

00 

so it is a proper model. If I X. = «> , then P(N(t) = 00 ) = 1 for 

i=l 1 

0 < t < °°, a degenerate situation. (Proofs are given in the Appendix.) 

The infinite event model can be thought of either as a model of physical 
reality or as a mathematically tractible approximate model when there is 
a large but finite number of events. By setting X^=0 , i > n + 1 , 

the finite event model is obtained as a special case of the infinite event 
model. Thus, without loss of generality, infinite event -models can be 
considered. (See, also, Theorem 4.9.) 

An Exponential order statistic model is characterized by its 

parameter set X = {X. , X , ...} . The only restrictions on X are 

0/1 2 . 

oo 

X. > 0 , i=l,2,..., and I X . < 00 . So the allowable parameter sets 

i=l 

for DET/EOS models include many, many possibilities. Some interesting 
special cases are: 

1. X^ = X^ , 1 < i < n (constant rates); 

2. X^ = £log(v/i) 9 1 < ± < [ v ] (logarithmic rates); 

3. X^ = aft 1 ,l<i< co ,0<3 < l (geometric rates); 

4. X_^ = £i 71 , l<i<°°,l<ri< o ° (power rates). 

Case 1 is the famous Jelinsky-Moranda [12] model. Case 2 is another 
finite event model. Case 3 was observed by Nagel [21,22] for rates 
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estimated during replicated-run software debugging experiments. Case 4 
follows a pattern observed by R. W. Phillips of IBM which is reported by 
Adams [1]: "the relative numbers of design errors having each possible 

rate was proportional to a particular inverse power of rates." (This 
relationship will be revealed more clearly in Section 7.) 

A parameter set A can be characterized by a cumulative function: 

Let L(A) equal the number of elements in {A. e A: A. < A} , 0 < A < 00 . 

l a, i — 

oo 

If A is an infinite set with E A . < 00 , then L(A) = 00 and it will 
* i=l 1 

be better to work with the complementary set: Let L(A) equal the number 

of elements in {A.eA:A.>A},0<A<°°. For the above four 

1 * 

examples, the (complementary) cumulative functions are: 


1. (constant) 


2. (logarithmic) 


3. (geometric) 

4. (power) 



where [x] - max {i : integer, i < x} is the greatest integer function. 

00 

00 r 

Note that E A. = 1 L(A)dA ; thus the set of permissible cumulative 
i=l 1 J 0 

functions is precisely the set of all integrable nonnegative decreasing 
step-functions with integer steps. Deterministic EOS models exist in 
one-to-one correspondence with these functions. This characterization 
will be extended in Section 3 to obtain a broader class of EOS models. 

The mean function, M(*) , of the counting process (N(t) , 0 < t} 

is defined as M(t) = EN(t) , 0 < t , and using the infinite analogue of 

( 2 . 1 ) , 


5 



CO 


M(t) - E(l^ l [0>t] (X^) 


' £ E( 1 [o,t] ( V> 


= E P(X. < t) 


( 2 . 2 ) 


i=l 


x - 


= E (1-e X i C ) 
i=l 


= / '• - u 


(l-e At ) (-dL(X)) 


The intensity function, m(*) , of the counting process is the derivative 
of the mean function: 


m(t) = M* (t) 


= Z 
i=l 


, -A.t 
A . e i 
1 


oo 



0 


A e Xt (-dL(A)) . 


(2.3) 


Note that 
at points 


m(0 is the Laplace transform of a measure with masses A_^ 

00 

X. , i=l,2,...; the total mass is Z X. , which is finite. 
1 i=i 1 


From uniqueness of the Laplace transform for probability distributions 
(see Feller [9, p. 430]), it follows that a DET/EOS model is completely 
determined by its intensity function m( • ) , or equivalently by its 
mean function M(.) . Of the four previous examples of DET/EOS models, 
only the first (constant rate) has a mean function with a nice closed 
form: M(t) = n(l-exp (-X Q t) ) . In effect, this gives an alternative defini- 

tion of the Jelinsky-Moranda model: It is a DET/EQS model with mean function 
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M(t) = n( l-expC-X^t) ) . Or, equivalently the J-M model is defined as a 
DET/EOS model with intensity function m(t) = nX Q exp(-X Q t). The reason 
for focusing on the mean and intensity functions will become obvious in 
Sections 3 and M. 

The EOS model is based on two basic assumptions: independence of 

times until occurrence of different events and Exponential distribution of 
these times. In the general context of software reliability growth these 
assumptions will not be exactly met; it can only be hoped that they are 
reasonable approximations. Independence can fail because of interaction 
between bugs; for example in experiments performed at the NASA AIRLAB 
facility, G. E. Migneault has observed instances of interaction, e.g. , a 
program containing two bugs can have a higher failure rate than the sum of 
the failure rates of the two corrected versions each of which contains only 
one bug. Exponential! ty can fail when the program input changes nonrandomly 
over time or exhibits a "memory." 

There exist several possible justifications for assuming Exponential 
distributions. In a carefully designed system occurrences of design flaws 
should be low probability events, i.e., "rare events." Poisson processes 
with their exponential waiting times are often good models for this type of 
phenomenon; see Breiman [ 3 ] and Ross [23]. Campbell and Ott [4] use EOS 
models for manifestation times of design flaws in nuclear reactors. A more 
rigorous justification for exponentiality can be given for programs that 
perform calculations based on individual discrete inputs; if a design flaw 
manifests itself for proportion p of the possible inputs and the successive 
inputs are chosen independently and randomly, the number of inputs until 
manifestation will have a Geometric distribution with parameter p. If 
execution times for each input are independent of success or failure of the 
computation and p is small, the Exponential is a good approximation (see 
Appendix). The Exponential will also be a good approximation for this type 
of program when successive inputs are not independent but are a regenerative 
process, e.g., successive inputs may correspond to a random walk through the 
space of all possible program inputs (see Appendix). 



Exponential Order Statistic models based on specific parameter 
sets are quite useful for specific applications: The Jelinsky-Moranda 

model is widely used; Migneault (experiments performed at NASA Langley 
Research Center's AIRLAB facility) has used the EOS model with geometric 
rates to predict successfully the occurrence of new, as yet unobserved, 
bugs. It is obviously a rich and useful class of models. However, it 
seems clear that parameter sets whose elements satisfy exact functional 
relationships as in the four examples (constant, logarithmic, geometric, 
or power relationships) can only be approximate. Thus, arbitrary 
parameter sets should be considered; this leads to nonparametric 
approaches: Campbell and Ott [4], for example. Also, stochastic 

parameter sets should be considered; this leads to alternative 
derivations of some existing software reliability growth models and 
additional insights. 

3. Doubly Stochastic Exponential Order Statistic Models 

Exponential Order Statistic models are based on the assumption 
that a given bug in a given program has a particular rate of occurrence, 
If the program is executed in a time-homogeneous environment the 
observed rate at which the bug causes errors will converge to X as 
execution time increases to infinity. By performing replicated-run 
software debugging experiments [21,22] the error rate for a particular 
bug can be estimated to any desired degree of accuracy. In this sense, 
for a given program in a time -homogeneous usage environment the parameter 
set Jl = » *2’ * s a set deterministic values. 

Now let use consider possible models for the parameter set X. In 
the previous section, various deterministic functional relationships were 
presented. If design errors are accidentally included in program, it is 
difficult to imagine that their rate would obey exact functional 
relationships. It is more plausible that the rates would be described by 
these relationships plus some randan noise. This suggests that X be 
modeled as a realization of a stochastic point process; for example 
{ ^ 1 » ^ 2 * •••} could be a realization of a nonhomogeneous Poisson process 
(NHPP ) . This gives a doubly-stochastic model: one process for the 

rates, and a DET/EOS process for occurrence times conditional on the 
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rates. It is a doubly-stochastic exponential order statistic (DS/EOS) 
model. 

Define A = (A , A , A , ...) as a point process on (O, 00 ) , 

t\j 1 Z J 

i.e., a collection of random points greater than 0 . We shall consider 
A as a random perturbation from a deterministic model: Let L(A) , 0 < A , 

Or 

be any nonincreasing function which decreases to 0 as A increases . 

Define A^ = L -1 (i) , i=l,2,3, . . . ; for example, L(A) = v exp(-A/0 , 0 < A 

gives the logarithmic deterministic set of rates from the previous section. 

From Figure 3.1 we see that a random perturbation from this deterministic 
model can be achieved by perturbing {l,2,...n} into the new values 

{u., , u 0 ,...u } and then defining A. = L"^(u.) , i=l,2,...n . The set 
1 z n ii 

{u. , u 0 ,...,u } can be the realization of any point process 
l 2 n 

U = (U , U , ) on (0,v) , but we shall only consider two possibilities 

Or 1 Z 

(a) U is the process of n independent Uniform [0,v] order statistics, 
and (b) U is a homogeneous Poisson process (HPP) with unit rate. For 

'Ip 

these cases we have the following general theorems (see Appendix for 
proofs) : 

Theorem 3.1 : Let L(A) be a nonincreasing function on [0,«0 

such that L(0) = n and lim L(A) = 0 . Let U = (U_ , U 0 , ...,U ) 

i z n 

A-* 50 

be n independent Uniform [0,n] order statistics. Define = L 1 (U i > , 

i=l, 2, . . . ,n . Then A = (A , A , . . . ,A ) are n independent order 

r\j l z n 

statistics from a distribution with complementary cdf L(*)/n and 
complementary cumulative mean function L(') . 

Theorem 3.2 : Let L(A) be a nonincreasing function on [0,°°) 

such that lim L (A) =0 . Let U = (U. , U_ , . . . ) be a homogeneous 

X Z 

X-XD 

Poisson process with rate 1. Define A^ = L ^(U^) for i such that 

U. < L(0) . Then A = (A , A . . . ) is a nonhomogeneous Poisson process 
i ^ 1 z 

with complementary cumulative mean function L ( * ) . 




we consider three 


Thus for a given function L(*) with L(0) < 00 , 
models for the rates {X , A 2 , ...}: (a) i.i.d. order statistics 

from 1 - L(*)/L(0) ; (b) NHPP with mean function L(0) - L(-) ; and 


(c) deterministic model. If L(0) = 00 , only cases (b) and (c) are 
possible and the NHPP must be described using the complementary mean 
function L(0 . It is hoped that looking upon these random models as 
perturbations of the deterministic models will provide some motivation 
for their consideration. Another approach would be simply to postulate 
the random models without mentioning the associated deterministic model. 
Regardless, they will be useful and interesting. 

The Jelinsky— Moranda model (constant rates) has L(A) = 


n > 0 < X ■ If L 
[X 0 ,") 

as the complementary cdf, G( •) 


is approximated by a smooth curve such 
, of a Gamma distribution (with mean A 


and large shape parameter) multiplied by n , we get a "noisy” J-M model; 
see Figure 3.2 , Following the above there are two options: A may be 

(a) n i.i.d. order statistics from G(0 , or (b) an NHPP with cumulative 
mean function n(l-G(-)) . Another simpler possibility for introducing 
noise into the J-M model is an NHPP with complementary cumulative mean 
function L(A) = n 1^ (A) , 0 < A , i.e., N rates all equal to 

Aq , where N has a Poisson distribution with mean n . 

We are considering i.i.d. order statistic (IIDOS) models and NHPP 

models for A . Another possible dichotomy among models concerns finite 

a, 

and infinite component A . This is only relevant for NHPP’s. It turns 

out to be unnecessary to make this distinction because NHPP’s can be 
defined generally regardless of whether L(0) is finite or infinite. 

The transformation of an HPP into an NHPP is illustrated in Figure 3.3 
for both cases. Thus we consider IIDOS models and NHPP models, each of 
the former having a finite deterministic number of bugs and each of the 
latter having either a finite (Poisson-distributed) number or infinite 
number of bugs. In all cases L (A) denotes the complementary cumulative 
mean function, i.e., the expected number of bugs which have rates greater 
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Figure 3.3 Nonhomogeneous Poisson processes: 
finite and infinite number of 
events 
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than or equal to X • 


The DS/EOS processes (N(t) , 0 < t} have especially nice forms 
for 1ID0S A’s and for NHPP A's : 

Theorem 3.3: Let A be an IIDOS process with cumulative mean 

function L(X) , 0 < X . Then, the DS/EOS process with rates A is 

— 'Xj 

CO 

an IIDOS process with mean function M(t) = f (1-e )dL(X) , 0 < t , 

co 

/ “X t 

gXe dL (X) , 0 < t . 

Theorem 3.4: Let A be an NHPP process with complementary 

^ 

cumulative mean function L(X) , 0 < X . Then, the DS/EOS process with 

oo 

/ “X t — 

n (l-e )(-dL(X)) , 

^ 00 U 

X t — 

q Ae dL(X) , 0 < t . 

Note that the above DS/EOS processes are completely specified 

bv whether A is IIDOS or NHPP and by the complementary cumulative 
J % 

mean function L(*) . They can be denoted as IID0S/E0S(L) and 

NHPP/EOS (L) , respectively. With this notation the above theorems can 
be summarized as 

IIDOS/EOS (L) = IIDOS (M) 

NHPP/EOS (L) = NHPP (M) , 

where M(t) is defined above. Thus by a rather circuitous, but 
physically meaningful, route we have arrived at some familiar models. 

Now consider doubly stochastic versions of the four examples 
from Section 2: 

(n , 0 < X < X 

1. L(A) =/ ; M(t) = n(l-e 0 ) 

(o , X Q < X < °° 

2. L (X) = v exp(-X/£) ; M(t) = v 



14 - 



3. L(A) = 


log A - log a 
log 6 


M(t) = 


_1 

log B 


oo 


E 

i=l 


(-at)* 

i-il 


4. L (A) = (C/A) 1/n ; M(t) = r(l-l/ n ) (5t) 1/n 

In case 1, IIDOS(M) is still the J-M model, but NHPP(M) is the Goel- 
Okumoto [10] model. In case 2, for integer v IIDOS(M) is a case of 
the Littlewood [16] model. Cases 3 and 4 are infinite event models and 
therefore are NHPP’s: Case 3 is apparently new. Case 4 is a familiar 

NHPP, sometimes referred to as the Weibull NHPP because its intensity 
function is identical to the Weibull hazard rate function; Crox^ [6] 
and Duane [7] have used it as a model of reliability growth. Thus 
DS/EOS models provide alternative derivations for some existing proven 
reliability growth models. 

4. Replication, Nondis tinguishability and Approximation 

We are studying three families of processes: exponential order 

statistic (EOS) models based on a deterministic parameter set 
A = {A- , A 0 , ...} , doubly stochastic exponential order statistic 

(DS/EOS) models whose parameter sets are order statistics of independent 
identically distributed (IIDOS) random variables, and DS/EOS models 
whose parameter sets are nonhomogeneous Poisson processes (NHPP) , We 
are interested in relationships between these families of models. In 
particular, we seek answers to the questions*: What role does replication 

play? In what senses can these families be distinguished from one 
another? In what senses can they be used as approximations to one 
another? Within individual families, do some models approximate others? 

The role of replication is important and needs clarification. 
There are two replication concepts: (a) separate programmers or groups 

of programmers can independently create different versions of the same 
program, and (b) a single program may be subjected to several separate 
debugging runs [8,21,22]. Figure 4.1 illustrates the program creation 


15 - 



Create Program: 
Realization of A 

— ^2 9 ••••} 

given program spec, 



B 


Run & Correct Program: 
Realization of 
{N( t) , 0 < t} given 
{ A.^ 9 A^ 9 


il 


Loop B: Replication of run 


Loop A: Replication of creation and run 


Figure 4.1 Different replication concepts 
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and running and the two replication possibilities: From a single program 

specification, the program is created (Box A) and then subjected to a 
debugging/correction run (Box B) . Replicated debugging/correction runs 
for a single program are represented by Loop B. Replicated program 
creation corresponds to independent passes through Box A. Replicated 
creation and a debugging/correction run is represented by Loop A. A 
nested design would involve several replicated runs for each replicated 
program, for example in Nagel’s [22] original pioneering experiment each 
program specification had two replicated programs and 50 replicated 
debugging runs, i.e., two passes through Loop A, each containing 50 
passes through Loop B. 

In this paper we are assuming certain stochastic models for what 
happens in Box A and Box B of Figure 4.1. In Box A a program is created 
containing bugs. In Box B, the program is run for a certain length of 
time; when a bug manifests itself it is corrected. It is assumed that 
the usage environment is time homogeneous throughout the experiment; so 
we assume that each bug has a fixed (unknown) deterministic manifestation 
rate associated with it. We assume three possible models in Box A: 
these rates { X ^ > . ..} are the realization of a deterministic 

process, an IIDOS process,' or an NHPP . Given the rates produced in Box A, 
the manifestation times {s^ , s^ , ...} generated in Box B will be the 
realization of a DET/EOS process with those rates. Thus, from Section 3, 
we see that a single pass through Box A and then Box B leads to one 
observation of a DET/EOS process, an IIDOS process, or an NHPP; this 
depends upon whether Box A is deterministic, an IIDOS model, or an NHPP 
model, respectively. This means that independent replications around 
Loop A of Figure 4.1 each involving a single run (one pass through Box B 
for each pass through Box A) will yield independent realizations of 
either a DET/EOS, IIDOS, or NHPP model. If a large number of replicates 
is taken, then by standard statistical consistency arguments it is 
theoretically possible to distinguish between the three models and to 
estimate the model parameters to any desired degree of accuracy. This is 
rather unrealistic in practice, however. Existence of a single replicate 
is probably more common and meaningful when EOS models are used. 
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Multiple replications can involve either replicated programs or 
replicated debugging/correction runs. For replicated versions of the 
same program it is unreasonable to model the error rates (A^ , > . . . } 

for different programmers as independent, identically distributed 
realizations of a single simple stochastic process; replicated-run 
experiments performed to date [8,21,22] suggest a possible noisy 
geometric pattern (Example 3 of Section 2) for different programmers but 
with distinctly different parameters. As Nagel [22] points out, there is 
a definite programmer effect. Thus for replicated programs (Box A of 
Figure 4.1) more complicated models are nedded for A . The IIDOS and 
NHPP models we use for A should be considered as a way of introducing 
noise into the pattern of error rates, not as a model for different 
programmers. (It might also be interpreted in terms of Bayesian subjective 
probability for the individual programmer.) If multiple debugging/correction 
runs are performed for a single version of a program, then we observe 
replicated sample paths of a DET/EOS process with a given A = (A.. , A 0 ,...} 

However, with such an experimental design it is more natural and efficient 
(for estimation of A) to identify the bug which corresponds to each failure 
observed. Thus the data will be collected as samples of manifestation times 
for each bug observed rather than sample paths of a DET/EOS process. This 
leads to different questions and analyses than those with which this paper 
is dealing. Thus for the scheme of Figure 4.1, we will avoid replication. 
Furthermore, and very importantly, most software reliability growth data 
will consist of a single replicate. Thus we shall have a single replicate 
of either a DET/EOS model, an IIDOS model, or a NHPP model. 

Let us consider the problem of distinguishing between an IIDOS model 
and a NHPP model by observing one realization of the process. Let F(*) 
be a cdf on [0,oo) and let (N^(t) > 0 < t} be the counting process for an 

IIDOS model with m observations, i.e., cumulative mean function 

M (t) = mF(t) , 0 < t , and occurrence times {S , S , ...,S } 

1 — i,x_L,z i,m 

Let {N^Ct) , 0 < t} be the counting process for an NHPP with cumulative 

mean function M^Ct) , 0 < t , and occurrence times {S^ ^ » S 2 2 ’ S 2 3 * *’ 
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There are several well-known relevant theorems: 


Theorem 4.1 : Suppose 3^(0 = Y F(t) 9 ® < t 9 where 0 < y < oo , 

Conditional on N o («0 = n , (S 0 - , S 0 n , . ..,S 0 ) are distributed as 

z z , 1 z, z z , n 

i.i.d. order statistics with cdf F(*) , i.e., (S„ , S 0 0 , . ..,S n ) 

z, 1 z, z z,n 

is an IIDOS process with mean function nF(-) . 

Theorem 4.2 : Let T be a deterministic time or a random time 

independent of (N^(t) , 0 < t} . Conditional on {T = s , ^(s) = * 

^ S 2 1* S 2 2 * ## ‘ ,S 2 n^ are distributed as i-i-d. order statistics with 

cdf G(t) = M^(t) /M^Cs) , 0 < t < s • 

Theorem 4.3 : Conditional on { S ^ n+ ^ = » ( S 2 1 5 S 2 9 ’ ’ #,,S 2 n^ 

are distributed as i.i.d. order statistics with cdf G(t) = M^CO/M^Cs) , 

0 < t < s . 

Theorem 4.4 : Let T be a deterministic time or a random time 

independent of (N^(t) , 0 < t} . Conditional on {T = s , N^(s) = n} , 

(S , S , . . . ,S n ) are distributed as i.i.d. order statistics with 
■L y -L -L y Z y n 

cdf G(t) = F(t)/F(s) , 0 < t < s . 

Theorem 4.5: Conditional on (S, = s) , (S.. , S, „ , . ...S, ) 

j i ) n * i ljl 1)Z l,n 

are distributed as i.i.d. order statistics with cdf G(t) = F(t)/F(s) , 

0 < t < s . 

Theorem 4.6 : Let T^ be a stopping time for {N^(t) , 0 < t} 

and let T ^ be the identically defined stopping time for {^(t) , 0 < t} . 

If l^Ct) = y F(t) ,0<t<s,0<y<oo, then (N^(t) ,0<t<s|T^ = s, 
N^(s) = n) and {^(t) , 0 <t<s|T 2 = s, ^(s) = n} are identically 
distributed. 
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The implication of the above theorems is this: If a single 

replicate of a reliability growth process is observed over [0,s] in 
such a way that the assumptions of the above theorems are met it will be 
impossible to distinguish between certain families of IIDOS models and 
related families of NHPP models. As an example, consider two sets of 
cumulative mean functions, 

M = : M (t) = m(l-exp(-A t) ) , m=0,l, 2, . . . , 0 < A < °°} and 

M 2 = : M 2 (t) = Y (1-exp (-At) ) ,0 <y <oo > 0 < A<° 0 }; and the 

two hypotheses H : IIDOS (M e M ) and H : NHPP (M e M ) , 

“L -L I 2 2 2 

which correspond to hypothesizing the J-M models or G-0 models 

respectively, mentioned in Section 3. Because the scale factors (m for 

and y for M are completely general it is impossible to 

distinguish between and based on the number of events in (0,s) , 

i.e., a single observation comes from : Binomial or Poisson; 

the only possibility is to look at the joint distribution of times of 
occurrence (which includes whether the stopping rule is satisfied) . 

But, by Theorems 4.2 through 4.5, for both and the occurrence times 

are i.i.d. order statistics from G(t) = (1-exp (-At) )/ (1-exp (-As) ) , with 
unknown A , 0 < A < 00 ; thus we cannot distinguish between and H 2 in 

the classical hypothesis testing sense if only one replicate is available. 

If a more general stopping rule is used Theorem 4.6 gives an analogous 

result. This example generalizes to include any family of cdf f s, 

F - (F : a e A) : let M = (M (•) : M (t) = mF (t) , m=0 , 1 , 2 , . . . ,a e A) 

a 1 1 l a 

and M 2 = (M 2 (0 : M 2 (t) = Y F a (t) ,0<Y<°°, ae A} > then 

H : IIDOS (M 1 e ) and H 2 : NHPP(M 2 e M^) cannot be distinguished by 

observing one replicate. 
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This nondistinguishability between families of IIDOS models and 
NHPP models is really equivalent to an important fact first noted by 
Langberg and Singpurwalla [14] within the context of reliability grox^th 
modeling: Theorem 4.1 has the interpretation that any NHPP with 

M(°°) < 00 is a mixture of IIDOS processes with a Poisson [mean M(<») ] 
number of observations from cdf F(t) = M(t)/M(°°) . The above non- 
distinguishability is thus equivalent to the problem of distinguishing 
between a constant population and a Poisson population with totally 
unknown mean when only one observation is available. Of course 
additional information may destroy the nondistinguishability: for 

example, bounds on the number of order statistics or the mean of the 
Poisson distribution may enable conclusions to be drawn from one 
observation. However, the situation arises frequently where parametric 
models are postulated with no constraints or information assumed for 
parameter values. (Theorem 4.1 also plays an additional unifying role 
if the distribution of N(°°) is considered subjectively for a Bayesian 
analysis; see Langberg and Singpurwalla [14].) 

In a weaker sense it is impossible to distinguish between 
deterministic and doubly stochastic EOS processes using a single 
replicate. For a given program it is possible to estimate the parameter 
set X = { , . . . } to any desired accuracy by replicated debugging 

(Loop B of Figure 4.1). However, even if the parameter set is completely 
known it still is a single replicate of the A-process and we are again 
faced with the problem of choosing between a deterministic model and a 
stochastic model when only one observation is available. If all possible 
deterministic models and all possible IIDOS and NHPP models for A are 

'V 

allowed then there is no statistical basis for choosing from one class 
rather than the other. If the choice is restricted to certain families 
of models, e.g., parametric families of deterministic or stochastic 
models of A , then it may be possible to distinguish between 

'b 

deterministic and stochastic. 

Now consider the very restrictive case where the cumulative mean 
function L(0 of the A-process is known, but the A-process may be 
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either deterministic or an IIDOS process; i.e., let Lg(0 be a known 

function such that L n (°°) = n+1 , A either is {A. = L^(i) , i=l,2,...,n} 

or is an IIDOS of n observations from F q(’) = Lg(0 /(n+1) . Note 

that the deterministic model is equivalent to sampling without replace- 
ment from a population with cdf = [ (n+l)F 0 (*) ]/n . Note that, 

if n is moderately large, and F^ are very close to each other. 

From a single observation of the failure process (N(t) , 0 < t} , 

it will be virtually impossible to determine whether A f s are drawn 
with replication from F^ or F^ . However, there will be some 

measure of distinguishability based on sampling with replacement (from F ) 

or without replacement (from F^) : both processes will fluctuate about the 

CO 

mean M n (t) = f (l-exp(-At) ) dL (A) , but there will be less variability 

in the process without replacement (the deterministic EOS process) . Thus 
in principle, a statistical test could be based on variation about M^(*): 

for small variation the deterministic model is preferred and for large, 
the IIDOS model. It is hard to imagine that such a test would have much 
power. Finally, a more realistic scenario would include a large para- 
metric family (not just one function), and furthermore the single 
replicate of the failure process would be observed only for a finite 
time interval, not in its entirety. This makes matters even worse and 
leads one to the conclusion that, in a practical sense, deterministic 
EOS processes and IIDOS/EOS processes that have a moderately large 
number of events and are based on the same parametric family of L(*) , s 
cannot be distinguished by observing a single replicate of the failure 
process . 

There are some theorems relevant to this discussion. Let L(-) 
be a function that decreases to 0 on (0,°°) such that 

M(t) = - / (l-exp(-At))dL(A) exists for 0 < t < 00 . 

Let {N iCt) , 0 < t} , k=l,2,..., be a sequence of independent 

U , 1C — 

deterministic EOS processes; the parameter set of the kth process is 
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{X i = L -1 (i/k) , i=l,2, . • . } . If L(0) < « , let {t^ fc (t) , 0 < t} 

be a sequence of independent IIDOS/EOS processes; the parameter set of 
the kth process consists of [kL(0)] i.i.d. order statistics from cdf 

F(A) = (L(0) - L(A)/L(0) . Let {N (t) , 0 < t} be a sequence of 

z ,1c 

independent NHPP/EOS processes; the parameter set of the kth process 
is an NHPP with complementary mean function kL(0 . 


Theorem 4.7 : For j = 0, 1, and 2, 


lim k^N. , (t) =M(t) , 0 < t < 00 j = 1 

k-K* J,k S 


Theorem 4.8: For j = 0, 1, and 2, {N. u (t) - M(t))/ \fk ~ , 0 < t} 

j 

converge in distribution to Gaussian processes as k-*» 


(The covariance structures of the limiting processes are different for 
j = 0, 1, and 2; see Appendix.) 

These theorems show the central role played by the mean function M(0 
for processes with a large number of events, regardless of the particular 
type of process. This plus the previously mentioned problems in distin- 
guishing between the three types of models suggests focusing on the mean 
function. Thus it might be useful to think of the mean function as the 
primary characteristic of the model and the particular type of model 
(DET/EOS, IIDOS , or NHPP) as a secondary characteristic. This is not 
a new idea; Musa and Okumoto [20 ] !, include M the Littlewood-Verrall model 
in their study by means of an NHPP with the appropriate mean function. 

We shall consider two models to be "close" to each other if their 
respective mean functions are close. The following limit theorems make 
this more precise. 
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, ...} be an infinite set 


Theorem 4.9: Let A = {A„ , A rt , A„ 

'v 12 3 

00 

of rates such that E 1, <.» . Let {N. , (t) , 0 < t} be a DET/EOS 

i=l u,lc 


k 

process with mean function II, (t) = E (1-exp (-A . t) ) , 0 < t < °° , 

k i=l 1 ” ' 

k=l,2,... . Then, as k » , {N^ k (t) , 0 < t} converges in distribution 

CO 

to a DET/EOS process with mean function M (t) = E (1-exp (-A . t) ) , 0 < t < 

i=l 1 


oo 


Theorem 4.10 : Let fc (t) , 0 < t} , k=l,2,... , be a 

sequence of IIDOS processes with mean function ( • ) , k=l,2,... , 

respectively. Suppose lim M (t) = M (t) < °° , 0 < t < °° . If 

k-*» k 

lim M (t) < °° , then the processes converge in distribution to an IIDOS 
t-*» 

process with mean function 11^(0 . If lim M^t) = 00 , then the processes 

t-KO 

converge in distribution to an NHPP with mean function M (•) . 


Theorem 4.11 : Let {N 2 k (t) , 0 < t} , k=l,2,... , be a 

sequence of NHPP's with mean functions M^*) > k=l,2,..., respectively. 

If lim M. (t) = M (t) < °° , 0 < t < oo , then the processes converge in 
k-Ho k 

distribution to an NHPP with mean function M (*) . 

CO 

It follows from Theorem 4.9 that the failure times of a DET/EOS 

process with a finite but large parameter set A = {A.. , A n , . ..,A } 

12 n 

can be approximated by those of a DET/EOS process with an infinite 
parameter set; this provides some justification for the infinite event 
models introduced earlier. Theorem 4.10 explains why both IIDOS models 
and NHPP models exist for bounded mean functions but only the NHPP exists 
when the mean function is unbounded. All three theorems show that 
closeness of mean functions implies close distributions of failure 
times . 
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For a given program the deterministic EOS model has certain 
physical motivation: the individual failure rates are physical 

quantities in the sense that they can be estimated to any desired degree 
of accuracy. The IIDOS and NHPP models are attractive because of math- 
ematical tractibility and successful application experience; however, 
they are somewhat more difficult to motivate and verify in a physical 
sense. The goal in this section has been to provide justification for 
these models on the basis of our inability to distinguish among them when 
a moderately large number of bugs are present and only one debugging 
replicate can be observed. (The argument is not as airtight as one might 
wish but does provide some evidence for the conclusion.) Thus we shall 
focus our attention on different reliability growth models as characterized 
by different mean functions, M(-) . (Equivalently, we can consider 
m(0 = M ? (0 , the intensity of the failure process, or L(-) , the 
complementary mean function of the A-process, or £(•) = - L T (*) , 

the intensity (if it exists) of the A-process. Any of the four 
quantities uniquely determines the other three.) 

5. Gamma Exponential Order Statistic Models 

The Gamma Exponential Order Statistic models (r/EOS) are a 
parametric family of Doubly Stochastic Exponential Order Statistic 
models; the intensity function of the parameter set A is 

a, 

A (JO = y A a-1 e" 3A , 0 < A , (5.1) 

co 

where a > -1 , 3 > 0 , and y > 0 . Recalling that L(A) =J*^£(p)dy 
equals the expected number of bugs with rate greater than A , we see a 
dichotomy: if a > 0 , then L(0) < 00 ; if -1 < a < 0 , then L(0) = <». 
Thus, for a > 0 , a finite number of events occur and A may be modeled 

as an NHPP with intensity £(•) . If a > 0 and L(0) is an integer, 

then A may instead be an IIDOS process (L(0) observations from density 

% _ 

f(A) = £ (A) /L(0) ) . When -1 < a < 0 an infinite number of events occur 
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and A will be an NHPP with intensity £(*) . (Throughout this paper 
the only stochastic models we consider for A are NHPP T s and IIDOS.) 

Now consider the failure process {N ( t ) , 0 > t} , its cumulative 
mean function 11(0 , and its intensity m(*)j from Theorems 3.3 and 3.4, 


m(t) = f Ae At £(A)dA 

*o 

OO 

f , -At ,01-1 -3A... 
= J Ae y A e dA 

0 


= y r(a+i)(t+e) 


(5.2) 


t 

M(t) = f tn(u)du 

J 0 

= y r(a+l) f (u+B) a ■''du (5.3) 

JO 


Y $ a r(a)(l-(t/e+l) a ) , a > 0 


Y log (t/g + 1) 


a = 0 


|^Y3“ a r(a+l)(-a) 1 ((t/3+l) a -l) , -1 < a < 0 . 

For a > 0 , 11(0 is proportional to a Pareto cdf; thus if A is an 

IIDOS process, then by Theorem 3.3 (N(t) , 0 < t) is an IIDOS process 

based on the Pareto distribution, i.e., the Littlewood [16] model. 

For -1 < ot < 00 f if A is an NHPP, then by Theorem 3.4 {N(t) , 0 < t) 

% “ 

is an NHPP with mean function M( •) : for a > 0 we might call it a 

Pareto NHPP; for a = 0 , we get the Musa-Okumoto [20] Logarithmic NHPP; 
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for -1 < a < 0 , we get something that could be called a Generalized 
Power Law NHPP. 

The closure of the V /EOS family consists of the limits obtained 
as a-*-l,a-*+w,B-*0,B->-+“,Y’ > 0 , or y +°° . By Theorems 

4.10 and 4.11 it suffices to consider limiting behavior of sequences of 
mean functions. Two possible degenerate limits are ( t) = 0 , 0 < t < °° , 

and M^Ct) = 00 , 0 < t < 00 , the former corresponding to no failures 

in [ 0 , °°) and the latter corresponding to all failures occurring at t = 0 . 
Nondegenerate limits will have a finite expected number of failures in a 
finite time interval, for example, M(t^) = n^ for some 0 < t^ < °° 

and 0 < n^ < 00 . Thus, to find nondegenerate limits, we can consider 

sequences of T / EOS mean functions (11^(0 s k=l,2,...} such that 

= n^ , k=l,2, ... . The mean functions in Equation (5.3) become 


r n 0 1 - (t/e + i) 

I 1 - (t^/3 + 1) 


a > 0 


M(t) 


n n log (t/B + 1) 
log (t Q /g + 1) 

Oj (t/g + 1) a -1 

(t 0 /e + D _a -i 

v 


a = 0 


-1 < a < 0 


(5.4) 


Without loss of generality let tg = 1 and consider the following limits: 
for 0 < t < 00 , 


lim (t/g + 1) “-1 = t' 

M) (1/6 + »-°-i 

11m (t/g + l)~ a -l = t 

< 1/6 + 


-1 < a < 0 


— 1 < a < 0 


(5.5) 


(5.6) 


lim log (t/B + 1) =1 (5.7) 

B-K) log (1/B + 1) 
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lim 

log (t/6 + 1) 

= t 





(5.8) 

3-x» 

log (1/6 + 1) 







lim 

1 - (t/6 + I)"" 

= 1 , 

0 

< 

a 

< OO 

(5.9) 

3-K) 

1 - (1/6 + l) _a 







lim 

1 - (t/6 + l) _a 

= t , 

0 

< 

a 

< OO 

(5.10) 


1 - (1/6 + 1)~“ 







lim 

(t/6 + l)" a -l 

= t , 

0 

< 

3 

< OO 

(5.11) 

a^i-1 

(1/6 + l)" a -l 







lim 

(t/6 + l)" a -l 

= log(t/3 + 1) 

, 0 

< 

3 

< OO 

(5.12) 

a^O 

(1/6 + l) -a -l 

log(l/8 + 1) 






lim 

1 - (t/6 + l)" a 

= log(t/6 + 1) 

, 0 

< 

3 

< OO 

(5.13) 

a^O 

1 - (1/6 + l)~ a 

log(l/6 + 1) 






lim 

1 - (t/6 + l)" a 

= 1 , 

0 

< 

3 

< 00 . 

(5.14) 

a-*° 

1 - (1/6 + l)' a 








(The above limits are verified by using fl^Hopital's Rule in various forms.) 

The NHPP with mean function M(t) = n^t a , -1 < ot < 0 , is the Power 

Law NHPP studied by Crow [6] and Duane [7]. The NHPP with mean function 

M(t) = n^t is the homogeneous Poisson process (HPP). The NHPP or IIDOS 

process with mean function M(t) - n *1 a degenerate case with all 

(finite number) failures occurring at t = 0 . In terms of models, the 
above limits have the following interpretations: 


Generalized Power Law NHPP (a, 3) 

Power Law NHPP (a) , as M 0 

Generalized Power Law NHPP (a, 3) 

*=*> HPP, as 3 00 

Musa-Okumo to Logarithmic NHPP (3) 

*=> Degenerate (0) , as 3 0 


(5.5a) 


(5 o 6a) 
(5.7a) 
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Musa-Okumoto Logarithmic NHPP (;3) 

=> HPP , as 3 00 

Littlewood's Pareto IIDOS model (a, 3) 

=> Degenerate IIDOS (0) model, as 3^0 

Pareto NKPP (a, 3) 

=0 Degenerate NHPP (0) , as 3 + 0 


Littlewood's Pareto IIDOS model (a, 3) 
=> HPP, as 3 00 

Pareto NHPP (a, 3) 

==> HPP, as 3 00 


Generalized Power Law NHPP (a, 3) 

=> HPP, as a ^ -1 

Generalized Power Law NHPP (a, 3) 

=t> M - 0 Logarithmic NHPP (3) > as o 0 

Littlewood's Pareto IIDOS model (a, 3) 

=> M - 0 Logarithmic NHPP (3) , as a. 0 


(5.8a) 

(5.9a) 

(5.9b) 

(5.10a) 

(5.10b) 

(5.11a) 

(5.12a) 

(5.13a) 


Pareto NHPP (a, 3) 

=> M - 0 Logarithmic NHPP (3) > as a V 0 

Littlewood's Pareto IIDOS model (a, 3) 

=> Degenerate IIDOS model (0) , as cx -*• 00 

Pareto NHPP (a, 3) 

=s> Degenerate NHPP (0) , as a 00 . 


(5.13b) 

(5.14a) 


(5.14b) 


The above results follow from Theorems 4.10 and 4.11 and the appropriate 
limits (Equations (5.5) through (5.14)). The relationship of the above 
models is summarized in Figure 5.1. Note that the Power Law NHPP is an 
NHPP /EOS process whose A-process has intensity 

1<X) - y , 0 < X (5-15) 
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HPP 


Jelinsky-Moranda 



a 


Figure 5.1 Gamma EOS models and limits: a parametric 

family that includes several well-known 
models. The parameters a and 8 take 
values in [-1,°°] x [0,co] . if (a-,g)-* 
C 00 , 00 ) but a/8 n , the parameterization 
is n instead of (a, 8) . The third 
parameter is always y , 0 < y . 


30 


IIDOS or NHPP 



for -1 < a < 0 . Note, however, that an HPP cannot be represented as a 
DS/EOS process because it shows no growth. The Degenerate (0) NHPP and 
IIDOS processes can be considered as EOS models if and only if A = 00 
is allowed in the definition of EOS models. 

Figure 5.1 shows that we have a nearly complete picture of T / EOS 
models and their limits, which are normalized so that M(tQ) = n o for 

some t^ and n^ . The only remaining case consists of limits when 

a 00 and 3 03 simultaneously. Suppose that 3 = ot/n , 0 < n < 00 ; 

then we have 


lim 

a-^° 


1 - (nt /a 4- 1) 

1 - (n/a + 1) 



9 


which, analogous to the above limits, is interpreted as 


(5.16) 


Littlewood's Pareto IIDOS ( a , a /n) model 

^ Jelinsky-Moranda (D) model, as & 00 

Pareto NHPP (a,a/n) 

=> Goel-Okumuto (n) model , as a -* «> . 


(5.16a) 


(5.16b) 


Thus the parametric family of T / EOS models and their limits is very rich, 
including some very well-known models as subcases. It might be fruitful 
to think of these models as special cases of the general three-parameter 
family, where a is a "shape" parameter, -1 < u < 00 , 3 is a "scale" 

parameter, 0 < 3 < 00 , and y is a "magnitude" parameter, 0 < y . 


Figures 5.2 through 5.11 show the mean functions for various 
(ct,3,y) in the parametric family of Figure 5.1. All mean functions are 
normalized so that M(100) = 100 . The purpose is to convey the extent 
and the richness of this family as well as to provide a visual comparison 
between the shapes of the mean functions for different subfamilies of the 
entire family. 
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MfT) 

0.00 40.00 S0.00 120.00 160.00 200.00 



1.00 40.00 80.00 120.00 160.00 200.00 

TIME 


Figure 5.2 T/EOS Mean Functions, Exponential 
Case (a/ g = n , a = °° , g = ») ; 
n =0 , .00808, .0167, .0265, 
.0385, .0550, .804, .126, .230, 
.600 
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120.00 


160.00 


200.00 


Figure 5. 


TIME 


r/EOS Mean Functions, Pareto Case; 
a = 16, B = °° , 2060., 977., 604., 
407., 281., 190., 119., 64.6, 24.3 





EOS Mean Functi 
= ■ 4 , g = <*> , 5 
4.0, 41.3, 24.9 






40.00 


80.00 


120.00 


160.00 


200.00 


Figure 5. 


TIME 


5 F/EOS Mean Functions, Pareto Case; 
a = 1 , 3 = “ , 203., 80.0, 40.8, 

22.5, 12.5, 6.67, 3.21, 1.25, .278 





40.00 


80.00 


120.00 


160.00 


200.00 


Figure 5. 


TIME 


6 r/EOS Mean Functions, Pareto Case; 
a = .25 , 3 = ® , 111., 36.9, 15.2, 

6.29, 2.37, .734, .157, .0169, .00034 






M(T) 

0.00 40.00 80.00 120.00 160.00 200.00 



1.00 40.00 80.00 120.00 - 160.00 200.00 

TIME 


Figure 5.7 r/EOS Mean Functions, Logarithmic Case; 
a = 0 , 3 = 00 , 80.4, 23.3, 7.64, 

2 . 17 , . 412 , . 0322 , . 000322 , 

.100 x 10~ 7 , .960 x 10“ 24 
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M(T) 

0.00 40.00 80.00 120.00 160.00 200.00 



TIME 


Figure 5.8 T/EOS Mean Functions, Generalized Power Case; 

a = -1/32, g = » , 76.6, 21.6, 6.77, 1.76, 

.272, .0123, .190 x 10 -4 , .345 x 10 -14 , 0 
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Figure 5. 


TIME 


9 r/EOS Mean Functions, Generalized Power Case; 
a = -1/8, 6 - ~ , 65.4, 16.7, 4.34, .732, 

.0308, .388 x 10 -6 , 0 






M(T) 

0.00 40.00 80.00 120.00 160.00 200.00 



Figure 5.10 r/EOS Mean Functions, Generalized Power Case; 
a = -1/2, B = " , 22.0, .835, 0 
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r/Eos 

3 = 0 
-.410 
-.046 


Mear 

> o 
, -.2 
. -.C 









The above limit properties, i.e., the continuity in a and g 
of the models in the above parametric family, have certain ramifications. 
In particular, if (a^ , g^ , y^) and ( a ^ , g^ , are close to each 

other the respective models with these parameters will produce failure 
data that appear to be similar. For example, if the true model is 
Littlewood's Pareto IIDOS model with very large a and g and a/g = n , 
because of statistical variation in the data, the Jelinsky-Moranda (n) 
model might often provide a better fit to the data; the converse is 
also possible. Another possible case of mistaken identity could involve 
Littlewood's IIDOS model with shape parameter a close to zero and the 
Musa-Okumoto logarithmic NHPP with the same value of g ; this case is of 
added interest because it involves a finite event model and an infinite 
event model. Typical failure data are over a finite interval; if a 
is close enough to zero it is quite possible that the infinite event 
H - 0 model would fit data from the finite event Littlewood model better 
than the Littlewood model itself. The continuity of the models around 
a = 0 means that there is no sharp demarcation between infinite and 
finite event models for failure data on a finite interval. This 
continuity and the common (a ,g , y)-parameterization of the models in 
Figure 5.1 suggests that it is appropriate to consider them as one model: 
Perhaps we should call it the J-M-G-O-L-M-O-D-C Model (Jelinsky-Moranda 
Goel-Okumoto-Littlewood-Musa-Okumoto-Duane-Crow Model) . 

6. Complete Monotonicity Property 

The DET/EOS, the IIDOS/EOS, and the NHPP/EOS processes are all 
characterized by their cumulative mean functions or, equivalently, by 
their intensity functions; i.e., the mean function M(-) , or intensity 

m(0 , and the family (deterministic, IIDOS, or NHPP) to which A 

% 

belongs completely determine the distribution of the failure process 
(N(t) , 0 < t} . For NHPP/EOS models, the class of possible intensities 

will be proven to be exactly the class of positive completely monotone 
functions such that lim m(t) = 0 and M(t) = f m(u)du < « , 0 < t < «> 

t~>°0 
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Intensities for DET/EOS models and IIDOS/EOS models are "almost dense" 
subsets of this class. 

First, the family of admissible L(*) T s must be characterized. 

Recall that L(A) equals the expected number of bugs with rates greater 
than or equal to A ; L(.) must be a nonincreasing function on [0,°°] . 

The following three theorems will characterize admissible L(.) f s: 

Theorem 6.1 : Let {A^ , A^ , be a deterministic process with 

_ 00 

complementary mean function L(*) ; then £ A . < « if anc j on iy if 

,p - 1-1 1 

lim L(A) =0 and l L(A) d A < » , 0 < y < » . 

A-**> JO 

Theorem 6.2 : Let , A 2 , . . . ,A } be an IIDOS process with 

n 

complementary mean function L(>) ; then P( £ A. <») =1 if and only 

i=l 1 

if lim L(A) = 0 . 

A-x» 


Theorem 6.3 : Let {A^ , > . ..,} be an NHPP with complementary 

_ 00 

mean function L(-) ; then P( £ A ± < » ) = 1 if and only if lim L(A) =0 

i=l X-KO 


M _ 

and f L(X) dA<°° > 0<y<c». 

J 0 


A proper EOS process must have £A ± < » ; if the A's are a realization 

of a stochastic process {A^ , A 2 > . . •} then EA^ must be finite with 

probability one. So Theorem 6 .1 through 6.3 lead to the following 
defini tion: 

Definition 6 . 1 : A complementary cumulative mean function L(-) of 

y 

a A-process is admissible if lim L(A) =0 , and f L(X)dX < ~ , 0 < y < «. 

A+0 j q 
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If the process is deterministic L(*) must be integer valued. For an 
IIDOS process L(0) must be integer-valued. (We use the convention 
that "admissible" refers to NHPP; restriction to deterministic and IIDOS 
models is discussed later.) 

Theorem 6.4: If L(‘) is admissible then 


_ w 

m(t) = - \ Xe“ Xt dL(A) < » , 0 < t < ~ . 

JO 


( 6 . 1 ) 


Furthermore, L(*) is admissible if and only if 




-At 


M(t) = -V (1-e ) dL(X) < oo , 0 < t < » , 


( 6 . 2 ) 


and lim M(t) = 0 . 
t->0 


Thus when L(-) is admissible the intensity and mean function are 
well defined and uniquely determine the distribution of the DET/EOS, 
IIDOS, or NHPP model. We wish to characterize the family of admissible 
intensities or the family of admissible mean functions. 

The definition of complete monitonicity and an important theorem 
relating completely monotone functions to Laplace transforms are given 
by Feller [9, p. 439]: 

Definition 6.2 : A function <)>(•) on [0,°°] is completely 

(n) 


monotone if it possesses derivatives <)>'(•) of all orders and 


(_l) n * (n) ( t ) > 0 , 0 < t < 


(6.3) 


Theorem 6.5 : The function <j>(') on [0,“] is completely monotone 

iff it is of the form 


Jo 


-Xt 


<f>(t) = l e dF(X) , 0 < t < - , 


(6.4) 
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where F is a measure on [0,°°] which may be finite or infinite. 

Comparison of Equations (6.1) and (6.4) suggest that we define 


F(y) 


AdL(A) , 

0 


0 < y < « , 


(6.5) 


which is the emulative function for a measure on [0,«>) ; integrability 
of (6.5) follows from admissibility of L(*) and the argument for 
integration by parts in the proof of Theorem 6.4 , Thus any admissible 
intensity function m(*) has the form of Equation (6.4) and must be 
completely monotone. Conversely consider any completely monotone 
function of the form (6.4) which is integrable in intervals around 0 : 
let 

00 > $(t) = f <j>(u)du , 0 < t < 00 

Jo 



00 

e _Xu dF(A)du 

0 


f F (0)du + ( C e -Au dF(A)du 

Jo Jo Jo 


( 6 . 6 ) 


= tF(0)+ j + A _1 (l - e At )dF(A) . 

Letting A \lF(A) = - dL(A) , we recognize Equation (6.2) when there is 
no atom at 0 , i.e., F(0) = 0 . Under this restriction. Theorem 6.4 
guaranties admissibility of L(-) and consequently $(•) is an 
admissible mean function. It is seen from Equation (6.4) and the monotone 
convergence theorem that F(0) = 0 is equivalent to lim = 0 . 

Summarizing gives the following theorem: t "*° 


Theorem 6.6 : A positive function m(*) is an admissible intensity 

(for an NHPP/EOS process) if and only if 
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(i) m(*) is completely monotone on [0,«?] 


(ii) lim m(t) = 0 , and 

t-x» 

(iii) l m(u)du < 00 , 0 < t < 00 . 

Jo 

Note that condition (iii) in the above theorem is automatically satisfied 
if m(0) < 00 and the counterexample m(t) = 1/t shows it to be necessary. 
Theorem 6.6 can be restated for the mean function. 

Theorem 6.7 : A positive function M(*) is an admissible mean 

function (for an NHPP/EOS process) if and only if 

(i) M(0) = 0 

(ii) lim M f (t) = 0 , and 

t-X>o 

(iii) (-l) n H (n) (t) < 0 , 0 < t < 00 , n > 1 . 

Now consider the families of admissible mean functions and 
intensities for the failure-time processes of DET/EOS and IIDOS/EOS 
models. These are subsets of the admissible functions of Theorems 6.6 
and 6.7 in a way very similar to that in which the integers are a subset 
of the real numbers. Properties of these subsets are given in the 
following theorems : 

Theorem 6.8 : If M(*) is an admissible mean function for an 

NHPP/EOS process with M(<») < co 9 then M^(t) = ([M(oo) 4- l/2]/M(<»)) M(t) , 

0 < t < 00 is an admissible mean function for an IIDOS/EOS process. 

Thus for every bounded NHPP/EOS - admissible mean function M(*) there 

exists an IIDOS/EOS - admissible mean function M^(*) such that 

sup[M(t) - M^(t) | < 1/2 . Furthermore, m(t) = M r (t) and m^(t) = M^’(t) 
t 

satisfy |m(t) - m^(t) | < (2et) , 0 < t < » . 
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Theorem 6.9 : If M(*) is an admissible mean function for an 

NHPP/EOS process whose A-process has complementary cumulative mean 
function L(-) , then the DET/EOS process with 

X = (X. = L~^(i - 1/2) , has cumulative mean function 

а, 1 

M^(*) which satisfies sup|M(t) - M^(t)| < 1 . Furthermore, 

m(t) = M f (t) and m^(t) = M^(t) satisfy |m(t) - m^(t) | < 2/ (et) , 

0 < t < 00 . 

These two theorems provide a sense in which the admissible functions for 
DET/EOS and IIDOS/EOS models are "almost dense" within the set of 
admissible functions for NHPP/EOS processes characterized in Theorems 

б. 6 and 6.7 . When the expected number of bugs is moderately large 
(50 or more, say), the above ± 1/2 and ± 1 terms seem negligible, 
obviating the need to make the restriction to the above subsets, 
especially when considering mean functions; thus we could use the 
characterizations in Theorems 6.6 or 6.7 for all three types of models 
(DET/EOS, IIDOS/EOS, or NHPP/EOS). 

If the failure rates of an EOS model are all less than or equal 
to l,i.e., 0 < X < 1 , then complete mono tonicity appears again in a 

different context. The result follows from material found in Feller 
[9, pp. 224-226]: 

Definition 6.3 : A sequence {c^ , k=0,l,2,...} is completely 

monotone if its differences alternate in sign, i.e., 

(-l) r A r c k >0, r > 0 , k > 0 . (6.7) 

Theorem 6.10 : The moments c^ of a probability distribution on 

[0,1] form a completely monotone sequence with Cq = 1 . Conversely, 
an arbitrary completely monotone sequence with c^ = 1 coincides 
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with the moment sequence of a unique probability distribution. 


From the Taylor series expansion for e we get 


Letting 


m(t) = Z (-f A 1 AdL(A) J (-t) 1 /!! , 

i=0 \ Jo / 

M(t) = Z /- f A 1 AdL(A) ) (-l) 1 t 1+1 /(i+l) ! 

1=0 \ Jo / 


.1 . 


- f A 1 AdL(A) / c , i > 0 , 

1 Jo 

c = - f AdL(A) , 

Jo 


we get 


m(t) = c Z (-1) 1 y . t 1 

i=0 -4 t- 


M(t) = c Z (-1) 1 p.t 1+1 

i=0 ik) T 


( 6 . 8 ) 


(6.9) 


( 6 . 10 ) 


( 6 . 11 ) 


( 6 . 12 ) 


Admissibility of L(*) implies that c < 00 ; y is the ith moment of 

a distribution on [0,1] with dF(A) = -AdL(A)/c . By Theorem 6.10 

(p , i=0,l,...} is a completely monotone sequence. Thus, the Taylor 
i 

series expansions in Equations (6.8) and (6.9) must have completely 

- + 

monotone coefficients whenever L(1 ) = 0 . Conversely, by Theorem 

6.10, if (v.) is a completely monotone sequence it must be a moment 

sequence of a distribution F(*) on [0,1] . If F(0) = 0 , then, as 
before, there exists an admissible L(*) such that 
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dF (A) = - aXdL(X) , 0 < X < 1 , and ip(t) = - Z (-1) X v. tVi .' 

a 1=0 1 
oo 

and H'(t) = — E (-1)^ V. t i+ ^/ (i + 1)! will be the intensity and 
a i=0 1 

mean function, respectively, of an EOS process. As before, lim ( t ) = 0 

will be necessary and sufficient to guarantee F(0) = 0 and consequently 
the existence of an admissible L(*) . In summary we have the following 
characterization: 

Theorem 6.11 : The intensity function and mean function of any 

EOS process with L(l + ) = 0 have the form 


m(t) = Z (- 1 )* v.tVi : (6.13) 

1=0 1 


M(t) = Z (-1) 1 v.t 1+1 /(l+l) I , (6.14) 

1=0 1 


respectively, where {v^} is a completely monotone sequence such that 


lim Z (-1) 1 v.t 1 /! I = 0 . (6.15) 

t^«> i=o 1 

Conversely, for any completely monotone sequence satisfying (6.15), 

the functions in (6.13) and (6.14) will be the intensity and mean function, 
respectively, of an EOS model whose rates are all less than 1 . 

By rescaling the time-axes (e.g., measuring time in nanoseconds instead 
of seconds), it is possible to satisfy the condition L(l + ) = 0 with a 
high degree of confidence. (However, if time is discrete and indicates 
the indices of repeated trials, then we may not be able to rescale and 
should perhaps consider a Geometric Order Statistic model; see Appendix.) 
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7. Additional EOS Models and Properties 

The general framework and description of EOS models presented 
above seems quite fruitful. It leads to new models, new perspectives 
on familiar models, and other insights. A miscellanea of such results 
is presented in this section. 

NHPP and IIDOS models . From Theorems 3.3 and 3.4 we know that 
IIDOS/EOS models and NHPP/EOS models are in fact IIDOS models and 
NHPP models, respectively. The converse is not true; not all IIDOS 
models can arise as IIDOS/EOS models, nor can all NHPP models arise 
as NHPP/EOS models. They must have differentiable cdf f s or mean 
functions, respectively, which satisfy the conditions of Theorem 6.7, 
or equivalently, densities or intensities which satisfy the conditions 
of Theorem 6.6 . This allows us to consider IIDOS models and NHPP 
models directly without considering the EOS structure. For example, 
let (N(t) , 0 < t} be an NHPP with intensity 

m^(t) = a(b + log(t+c+l)) ^ with 0 < a, b, c, d ; m^(-) satisfies 

Theorem 6.6, and therefore {N(t) , 0 < t} may arise from an EOS 

2 

model. On the other hand, suppose m^Ct) = 1/ (x + 2x + 2) ; this 

appears to be a plausible intensity, however the fourth derivative is 
negative for small t , violating complete monotonicity. Thus if we 
believe the EOS modeling paradigm we would not consider an NHPP with 
intensity n^O) as a possible model for the failure time process. 

Feller [9, p. 441] presents some properties which are useful for 
identification of completely monotone functions. 

Weibull IIDOS models . The Weibull distribution has cdf 

F(x) = 1 - exp (- (x/ g) a ) , x > 0 , (7.1) 


and density 


f(x) = ag a x a 1 exp(-(x/g) a ) , x > 0 


(7.2) 
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where a > 0 and 3 > 0 . The density f(-) satisfies Theorem 6.6 
(or, equivalently, the cdf F(*) satisfies Theorem 6.7) if and only 
if 0 < a < 1 , independently of the value of 3 . Thus, there exist 
IIDOS/EOS models whose failure time processes are Weibull IIDOS with 
shape parameter 0 < a < 1 . The mean function for the underlying 
A-process satisfies 


na3 a t a-1 exp(-(t/ 3 ) a ) 


Ae U dL(A) 

0 


(7.3) 


Solving (7.3) for L(*) is equivalent to inverting a Laplace transform; 
the case a = 1/2 can be found in tables of Laplace transforms? 


s exp(-ks 1 ^ 2 ) = j* e ^ S (ttA.) 1 ^ 2 exp (-k 2 /4A)dA (7.4) 


which gives 


AdL(A) = nl/ 23 _ 1 / 2 ( 7 rA )~ 1/2 exp(-S/4A) 


(7.5) 


Thus the A process is an IIDOS process from a distribution with density 

g(A) = (4 tt 3)~ 1/2 A _3/2 exp(-3/4A) . (7.6) 

If a £ 1/2 , there appears to be no closed -form solution for dL(0 
in Equation (7.3); however, this does not prevent use of the Weibull 
IIDOS model for (N(t) , 0 < t} . Thus we can identify possible models 
for the failure time process of EOS models without detailed knowledge 
of a model of the underlying A-process. Musa and Okumoto [20] 
introduce an NHPP with intensity proportional to f(-) of Equation 
(7.2). By the above analysis this is an acceptable model for the 
failure times of an EOS process, whenever 0 < a < 1 . This provides 
some additional motivation and justification for Musa and Okumoto to 
consider such a model. 

Gamma IIDOS and NHPP models . The Gamma distribution has density 

f(x) = -yx” 1 e -X ^ B , 0 < x , (7.7) 
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with 0 < a and 0 < 8 . If 0<a<l,f(*) is completely 
monotone and therefore by Theorem 6.6, 6f(«) can be the intensity 

of a failure-time process from some EOS model. 

Another General Family of Models . The above Weibull and Gamma 
models are special cases of failure-time models with intensity 

m(t ) = yx a 1 exp(-(x/g) 5 ) , 0 < t , (7.8) 


where 0<a<l t 0<8»0<6<l. This function satisfies Theorem 
6.6; thus the IIDOS process or NHPP with intensity m(*) can arise 
from an EOS model. 


Geometric failure rates. The deterministic model 


X = (X. = ctg 1 , i=l,2, . . . } 


(7.9) 


with 0 < 3 < 1 has been suggested by Nagel [21,22]. We now consider 
NHPP models for A which will show an approximate geometric pattern. 

'X. 

The analysis in Section 2 suggests an NHPP with 


L(X) 

because L“^(i) = ag 1 

MX) 


(log X - log a) /log g 0 < X < a 
0 a < X < 00 


(7.10) 


i=l,2,... . This NHPP A-process has intensity 

! -(X log B) _1 0 < X < a 

0 a < X < ® < 7 - U) 


The intensity and mean of the failure-time process are 


m(t) = (1-e at )/(t log (1/P)) , 0 < t 

oo 

^ -(~«t) , 0 < t 


M(t) = 


log(l/g) 


i=l 


(7.12) 
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and by Theorem 3,4 the failure-time process (N(t) , 0 < t} is an 
NHPP . 

Other methods of adding noise to the model of Equation (7*9) are 
suggested by considering 


-log X = {-log X = -log a - i log g , i=l, 2 , . . . } , 


which is a renewal process starting at -log a and having constant 
interarrival times equal to -log g . Using random i.i.d. interarrival 
times with mean -log g transforms this into a stochastic model. If 
i.i.d. exponential interarrival times are used, -log A is an HPP and 

A is the same NHPP as above, with the intensity of Equation (7.11). 

If a general renewal process is used for log A , the distribution of the 
failure-time process (N(t) , 0 < t} appears intractible. 

The T/EOS model with a = 0 , i.e., the Musa-Okumoto Logarithmic 
NHPP, may also serve as a model when the underlying failure rates are 
approximately geometric. Note that the intensity of Equation (7.11) 
and the intensity of the M-0 process, Equation (5.2), are both 
proportional to t ^ for large t . This implies roughly the same 
pattern of failure rates among the later-occurring bugs, i.e., the bugs 
with small rates. More precisely, for the M-0 model 


OO oo 

L(A) = j Jt(y)dp = j* yu -1 


e ^du = y 


£ " 


-abv, 
e dv 


OO J 

= - y(-5772157 + log 6A + Z ) . 

i=l 1,:L ' 


(7.13) 


As A -> 0 , L(A) behaves like - y log BA ; thus A^ = L '*'(i) shows a 

geometric pattern for large i . This behavior In connection with the 
frequent occurrence of geometric rates in replicated run experiments 
[8,21,22] suggests that the Musa-Okumoto Logarithmic NHPP will often be 
a good model; this is verified in their study using real data [20]. 
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Logarithmic failure rates . Example 2 of Section 2, 

A - {A. = £ log (v/i) , 1 < i < [v] } , showed a logarithmic pattern. 

^ x 

This can be generalized to 

A = { A . = £ log (v/ i)^ a , 1 < i < [ v] } , (7.14) 

^ x - - 

where 0 < a . These rates satisfy A^ = L”'*"(i) , where 
“ ct 

L(A) = v exp(-(A/£) ) , which is proportional to a Weibull survival 

function. The stochastic version will have rates {A. , . . . ,A } 

1 n 

which are Weibull IIDOS, or NHPP with mean function 
M(t) = V (1— exp(-A/£) a )) • 

There are three different EOS models related to the Weibull 

distribution: In the above process the failure rates are Weibull 

distributed with shape parameter 0 < a < 00 . In a previous example 

the failure times were Weibull distributed with shape parameter 

0 < a < 1 . Finally the Power Law NHPP is sometimes referred to as 

—6 

a Weibull NHPP because its intensity, m(t) = yt , equals the hazard 
function of a Weibull distribution with shape parameter a = 1-6 , 

0 < 6 < 1 . 

Phillips law . Phillips (see [1]) observed that "the relative 
numbers of design errors having each possible rate was proportional 
to a particular inverse power of the rates." This is equivalent to 

cx-l 

£(A) = yA , -1 < a < 0 , in our terminology, and gives the Power 

Law NHPP. In this case L(A) = (y/a)A a and a deterministic model has 

A^ = (ai/y) 1 ^ 06 , i=l,2,... (Example 4 of Section 2). Note that the 

plot of {(log i, -log A^) > i-1,2,...} is linear with slope 

1/a , -1 < 1/a < 0 . If A is used as a measure of the "size" of 
the design error, then we have a generalization of Zipf T s Law 
[26, pp. 195-202]. Zipf f s Law applies when the slope equals -1 , 
which corresponds to a = -1 ; however, when a = - 1 , A^ = y/ai and 
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oo 

E X . = 00 and a proper EOS model does not exist. Note, however, from 
i=l 1 

Section 5 that a suitably normalized Power Law NHPP converges to an HPP 
as a -1 ; so in this sense Zipf f s Law for failure rates gives an 
HPP for failure times, i.e., no growth. 


Effect of previous debugging. A design flaw with failure rate X 

— 6 X 

will remain undetected during [0,6] with probability e . Thus, if 
Lq(*) is the complementary mean function of the A-process for a 

program with no previous debugging, and if L^ ( • ) is for the A-process 
of bugs remaining after debugging for 6 time units, then 


dL 6 (X) = e 6X dL Q (X) , 0 < X . 


(7.15) 


Let iHq(*) and m^(*) be the intensities of the respective failure 
time processes, then 


V 


c 

■-J. 


Xe" Xt dL-(X) 

0 


Xe' X(t+S) <iL 0 (X) 


(7.16) 


= m Q (t+6) , 0 < t , 


(as one would expect). Since the choice of a time origin may be somewhat 
arbitrary, it is reassuring that models for different choices of origin 
are simply related as in Equations (7.15) and (7.16). In the r/EOS 
models, the parameter 3 could be interpreted as an indicator of 
something equivalent to previous debugging. The Generalized Power Law 
NHPP can be interpreted as a Power Law NHPP that has undergone previous 
debugging. A family of parametric models can be constructed using 
Equation (7.15) or (7.16) from a single EOS model. The family of 
r/EOS models is closed under such construction; this should be considered 
a desirable property for a parametric family. Note that the NHPP of 
Equations (7.10) through (7.12) does not have this property. An enlarged 
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family with 


*(A) = 


■! 


i oX -i -fix 
-(A log 6) e 


0 


0 < A < a 

a < A < °° 


(7.17) 


will have the property. But for large 6 , the truncation at a becomes 

negligible and we have the Musa-Okumoto Logarithmic NHPP again, of course. 

Truncated models for A . Another modification that can be made 


to any model is truncation of the A-process at A : 

max 


! L(A) - L(A ) , 0 < A < A 

- - max 

0 A < A 

max - 


trunc 


(7.18) 


Also, inadmissible models can be made admissible by truncating; for 
example, 


*U) = 


{ 


yA 


o-l 


0 < A < A 
- - max 


A < A 
max - 


(7.19) 


is an admissible intensity for the A-process for -1 < a < °° , but the 
nontruncated version is admissible only for -1 < a < 0 . 

Superposition of models . The superposition of two or more EOS 
models is again an EOS model. A program might have two main subparts and 
an interface between them, all written or designed by different people. 
Bugs can be identified as belonging to one of the three parts; this 

gives three failure-time processes: (N^t) , 0 < t} , {N 2 (t) , 0 < t) 

and {N^Ct) , 0 < t} . If each of the three is an EOS process then 

N 1 + N 2 + N 3 is 3 fai l ure-t i me process of an EOS model. In general, for 

^ — ^2. ’ ^2 ’ * * * ’ bet N^(t) = 1 1- q j.j (xp , i=l, 2, . . . ,n ; then 
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n n 

N(t) = E N.(t) = £ l rn , (X. ) , 0 < t , is the general deterministic 

i=l 1 i=l L °’ J 1 

EOS model of Section 2. Applying Theorem 4.9 gives that superposition 
of DET/EOS models, each consisting of one bug, are dense among all 
DET/EOS models. Thus any reasonably rich family of EOS models that is 
closed under superposition of its members and limits of its members will 
contain all EOS models. The T/ EOS family is not closed under super- 
position, but it may still be a family that is rich enough for many 
applications . 

Interfailure time models . Interfailure time models assume that 
successive interfailure times are independent (but not identically 
distributed). The Littlewood-Verrall [18] model is an example. It can 
be shown that the Jelinsky-Moranda model has independent interfailure 
times. It can also be shown that any EOS model with three or more bugs 
and nonconstant failure rates cannot have independent interfailure times. 
Thus, except for the overlapping J-M model, the EOS models and interfailure 
time models are distinct classes of models. Nevertheless, certain EOS 
models do suggest possible interfailure time models. For example, the 
Musa-Okumoto Logarithmic NHPP has M(t) = y log ( t / 3 +1) , so the ith 
failure will occur somewhere around s^ = M ^(i) = 8 (exp (i/y)-l) . 

They show that the interfailure time S . - S^ has a Pareto distribution 

conditional on S . = s : 
i 

P(S., 1 - S. > t|s. = s) = (t/s+B) + 1) _Y , 0 < t (7.20) 

Since S_^ is approximately equal to , then 

P(S., 1 - S. > t) 3 j (t/s.+B) + 1) -T , 0 < t , (7.21) 

and we postulate an approximate interfailure time model with Pareto 
interfailure times; the parameters of the ith interfailure time are 
y and 
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( 7 . 22 ) 


= s i + 3 = B(exp(i/Y)-1) + B = Be l/Y , i=1,2,... 

The Li ttlewood-Verrall is exactly such a model, having independent Pareto 
interfailure times: 


P(S i+1 " s i > = ( t/ip ± + 1)~“ (7.23) 

with t increasing. They used ip. = B- + B„i and = r + 8 i 2 

j. l i 2 i 1 2 # 

The above admittedly rough analysis suggests that ^ = b i B 2 i be tried. 
8. Inference and Prediction 


Software reliability growth models are useful in assessing software 
reliability; but creating models is only one step of the process. A 
typical scenario involves observing failures and correcting design flaws 

at times s^ , s 2 s n(t) during [0,t] and then trying to predict 

behavior during [t,»); quantities that are of interest include the 
current failure rate, the distribution of time until the next failure, the 
.number of failures over a finite horizon, etc. Littlewood (et al, 
Evaluation of Competing Software Reliability Predictions , Centre for 
Software Reliability Research Report, The City University, London, 

March 1985) describes a "prediction system," which involves three 
components: 


(i) Probability models that completely specify the 
distribution of failure times (S 1 , S 2 , ...); 


(ii) An inference procedure for picking a specific 
single model for particular observed data 


(Sj* s 2 , 


” S n(t) ); 


(iii) A prediction procedure that combines (i) and (ii) 

to give probability statements about future failure 

tlme3 (S N(t)*1- S N(t).2> •••>• 
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Our focus now is on part (i) of Littlewood’s Triad; this paper’s 
main concern is a particular class of probability models: EOS models. 

Nevertheless there are some observations pertaining to inference and 
prediction for EOS models that are worth making at this time. We shall 
make these briefly; more extensive study will appear in future research 
papers. 

The family of EOS models is very rich. The possible patterns 
of underlying failure rates are unlimited. The above inference and 
prediction steps will be easier if the class of possible models can be 
restricted a priori, i.e., before testing or, ideally, before the program 
is written. After many controlled software experiments are conducted and 
a large amount of field data are collected it may be possible to conclude 
that particular types of software tend to exhibit characteristic patterns 
for the failure rates. A geometric pattern X = {aft 1 , 0 < i} is 
emerging in the programs studied in certain replicated run experiments 
[8,21,22]. Phillips observed a different pattern in some IBM software 
[1]. Perhaps various software metrics [11] can be used to discriminate 
between classes of software that show different patterns in their 
underlying failure rates. A Bayesian analysis based on an a pviovi, 
distribution over the entire class of EOS models would be very desirable^ 
it is not clear at this time how to describe such a distribution. The 
point is that the richness and generality of all possible EOS models 
seem to suggest the need for a capability to base reliability predictions 
on more than just observation of past failures of the specific program 
in question. 

On the other hand, the r/EOS family of models may be rich enough 
to represent adequately the entire family of EOS models. Further study 
is required to confirm this, but it does look promising. The Tf EOS family 
includes both the geometric pattern and the pattern described by Phillips. 
It also includes as special cases several well-known models. Possible 
advantages to considering the single family parameterized by (a, 3, y) 
instead of considering the separate models should be explored. A 
Bayesian approach with this three-dimensional parametric family should 
be pursued. 
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The richness of the EOS family of models suggests that a non- 
parametric approach to inference and prediction be tried. The complete 
monotonicity properties of Section 6 and Theorem 4.7 reveal an approach: 
Estimate M(t) , 0 < t , with the curve satisfying Theorem 6.7, which 
is "closest" to the observed realization of (N(t) , 0 < t} . Or one 
can estimate the intensity of (N(t) , 0 < t} from the observed sample 
path using any of several nonparametric estimates and then find the 
"closest" function that satisfies Theorem 6.6 for an improved estimate. 

The first step in this direction has been taken by Campbell and Ott [4]; 
they used isotonic regression methods to estimate the intensity function 
of EOS failure processes. Miller and Sofer [19] explore the more general 
problem of regression with completely monotone functions. 

The nondistinguishability between NHPP and IIDOS models discussed 
in Section 4 has some ramifications for inference and prediction. If 
two alternatives and are statistically indistinguishable given 

the data, the question of "goodness-of-f it" is moot. However, in the 
case of NHPP versus IIDOS inference will differ: the likelihood functions 

in the two cases are different and so are admissible parameter values, 
e.g., M(°°) must be integer -valued for IIDOS models but not for NHPP 

models. Thus we may get different predictions using the two alternatives 
to which we are indifferent as far as any goodness-of-f it criteria are 
concerned. Since "quality of prediction" is of primary importance, the 
indifference between NHPP and IIDOS models in the first stage (i) of 
the prediction system may not carry over to the third stage (iii) . 

9. Conclusions 

This paper has presented EOS models and some of their properties. 
The idea of nonidentically distributed independent Exponential random 
variables is, of course, not new. However, when doubly stochastic EOS 
models are considered and when only a single sample path is observed, 
some interesting properties follow that are relevant to the study 
of software reliability growth. 

Exponential Order Statistic models are useful for several reasons. 
The structure of the models reflects to some extent the actual physical 
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failure process: the parameter A has a physical intepretation, and 

the doubly stochastic models distinguish between randomness of the 
failure rates and randomness in the usage environment. This may 
eventually allow deeper penetration into the mysteries of real failure 
processes; EOS models seem to be natural models for replicated-run 
software debugging experiments. The fact that several well-known software 
reliability growth models can be derived as special cases of EOS models 
is useful because the alternative derivation provides additional support 
for the validity of those models. Finally, the EOS paradigm is a 
framework within which new parametric families of reliability growth 
models can be identified (e.g., Section 7). 

A general adoption of EOS models leads to several new possibilities 
and points of view concerning inference and prediction. The mean M(*) 
and intensity m(.) of the failure process play a central role, as does 
the process of failure rates, A ; the relationship between these 

quantities is that of a Laplace transform, a connection which might be 
exploitable. The T / EOS model unifies several previously unrelated 
models; a unified approach to inference and prediction for these models 
based on the F/EOS model should be useful. The richness of the family 
of EOS models and the possibility of superposition of failure processes 
tend to challenge the ability of a parametric family to model all 
possibilities adequately. Nonparametric approaches can be based on 
complete monotonicity properties. Development of a Bayesian analysis 
encompassing the entire EOS family is worth pursuing. Lastly, the EOS 
family gives a diverse set of models from which data can be simulated 
in a systematic way and used as test data for evaluating various 
proposed inference and prediction methods. 

It is hoped that this paper will stimulate interest that will lead 
to further use and investigation of these models. 
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APPENDIX 


This appendix contains background information, proofs and 
additional discussion, Ross [23,24] provides a good introduction to the 
necessary stochastic processes, especially the Poisson process. Karlin 
and Taylor [13, Chapter 13] provide an excellent discussion of uniform 
order statistic processes, homogeneous Poisson processes and their 
relationship to one another. Cinlar [5] gives an excellent discussion 
of Poisson processes, especially the nonhomogeneous (nonstationary) 

Poisson process, which he shows to be a transform in time of a homogeneous 
Poisson process [5; Chapter 4, Section 7]; this leads to generalization 
of the results in Karlin and Taylor to nonuniform order statistics and 
nonhomogeneous Poisson processes. 

In Section 2 a proper DET/EOS model is defined as one for which 

CO 

1 X ^ < 00 . Recall that ^ Exponential (A^) , i=l,2,... , are 

independent random variables. The event {N(t) < «>} belongs to the 
a-field generated by {X^ , X^ , ...) , i.e., occurrence or nonoccurrence 

is completely determined by the values taken by X^ , X^ , ... . Further- 
more, the event {N(t) < °°) is independent of {X- , X 0 , ...,X } for 

12 n 

each n , n < 00 . Thus by the Zero-One Law for Tail Events [9, p. 124], 
either P{N(t) < °°} = 0 or P{N(t) < «} = 1 . When ZX ± < °° , 

minX^ ^ Exponential (EA^ ; thus P{N(t) = 0} = exp(-EA i > > 0 and the 

latter possibility holds: P{N(t) <«}=!. Conversely, if P{N(t) < «} = l 

then for some n < 0 ° , P{N(t) = n) > 0 , which implies that there is at 

least one subset of the integers {i- , i 0 , . . . , i } = I (t) such that 

1 / n n 

P({X < t , i e I C-t) } 0 {X. > t , i $ I (t) }) > 0 , which in turn 
1 — n 1 n 

implies P(X. > t, i£l(t))>0 which equals exp(-E A.) . Thus 
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if < °° , for i e I (t) , we get Z X. < <*> . This proves (under 

i=l 

the assumption that A^ < « , i > 1) that (N(t) , t < »} is non- 
00 

degenerate iff Z A . < °° . If A. = ~ for i=l,...,k and A. < « 
i=l 1 1 1 

for i > k , then N(0) = k and N(t) < 00 with probability one iff 

00 

E A . < 00 . 
i=k+l 1 

The Exponential approximation to the Geometric mentioned in Section 
2 is well known. The formal statement in terms of a limit theorem is: 

Let X n , n=l, 2, . . . , be a sequence of Geometric random variables with 

parameters p , n=l,2,..., respectively, i.e., P(X > i) = (1 - p ) X , 
n n n 

i=0,l,2,... . If lim P n = 0 , then lim P(p X > y) = e ^ . Thus 

n-*» U n-x» n n 

for small p the Exponential distribution can be used as an approximation 
to the Geometric distribution. 

Another situation wherein the time until failure is approximately 
Exponential involves regenerative processes. (See Cinlar [5] or Ross [24] 
for background.) Successive inputs to a program may not be independent 
but may instead exhibit dependence; for example, the sequence of inputs 
could be a random walk on the space of all possible inputs. More 
generally, the sequence of inputs could be a regenerative process. Such 
processes have a structure which consists of a sequence of independent 
identically distributed sojourns from a single state, which is called a 
regeneration point. If on any given sojourn the probability of 
experiencing the failure is p and the failure first occurs on sojourn N , 

then P{N = n} = p(l-p) n ^ , n=l,2,... . The duration of sojourns 1 

through N-l will be i.i.d. random variables T, , T 0 , . . . ,T„ - . If 

12 h-1 

ET < 00 , it can be shown using Laplace transform arguments that, as 

n-1 

p “* 0 , the distribution of p E T. converges to an Exponential 

i=i 1 
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distribution with mean ET. Thus if the duration in the Nth sojourn 

n-1 

until failure is negligible compared to E T. and p is small, the 

i=l 1 

distribution of time until failure will be approximately Exponential. 

Theorems 3.1 and 3.2 are well known. Theorem 3.1 deals with 
transforming i.i.d. uniform order statistics into i.i.d. order statistics 
of an arbitrary probability distribution using the "inverse probability 
transform;" see Law and Kelton [15, pp. 242-247] or Schmeiser [25]. 

Theorem 3.2 deals with transforming an HPP into an NHPP; see Cinlar 
[5, pp. 94-101] for a proof. 

Proof of Theorem 3.3 : Consider the unordered failure rates 

( A 1 * a 2 >--*> A n ) where n = L(°°) : P(A 1 < , A 2 - A 2 * * ,A n - X x? = 

n 

It L(A.)/n . Let T.(A.) be an exponential random variable with rate A. 
i=i 1 1 1 

OO OO , 

thus P(T . (A . ) < t) =-• f P(T . (A . ) < 1 1 A . = X) dL(X)/n = f (l-e _At )dL(A) /n 

11 Jo 1 1 1 JO 

The (T - (A, T (A ) ) are independent because the (A- ,... ? A ) are 
i ± n n In 

independent; thus we have a random sample of size n from cdf 

oo 

F(t) = f (l-e ^ t )dL(A)/n and the ordered T.(A.) f s are an IIDOS process 

Jo 1 1 

n 

The mean function is M(t) = EN(t) = E( 2 1 -,/t.(A.))\ 

i=l 11 J v 1 1 ) 

n 00 , 

= Z P (T (A ) < t) = f (l-e' At )dL(A) . 

i=l 1 JO 

Proof of Theorem 3,4 : Consider the Poisson process of (ordered) 

failure rates: A 1 > A 2 > A^ > . . * . This process has complementary 

cumulative mean function L(«) . Now consider 0 = t^ < t^ < t^ <... < t^ 

and split the above process into k processes: , N^ , . . . ,N^ . 
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If = A^ then this failure rate is put into the jth process with 
probability e 1 J -e 1 J , j=l,2,...,k . By well-known splitting 


theorem for NHPP's, are independent Poisson processes 

with EN . (°°) = 1 

1 J( 


•°° -At., -At. 

(e - e "*) (-dL(A)) . But note that the 

0 


splitting is equivalent to observing T^Ap , i=l,2,... : if A = A ^ 

-A.t. - -A.t. 

then P^^ < ^(Ap < tij | A ± = A ± ) = e 2 -e 1 3 . 

oo 

So N (oo) = z 1/ . (T. (A.)) , j=l,2,...k . But from above, 

i=l U j-1. P 1 X 

Nj (oo) , j=l,2,...k , are independent Poisson random variables with 


£ "At.: 

E( E N (-)) = V (1 - e 2 ) (-dL(l) ) 
j=l 2 Jo 


. Thus {T i (A ± ) , i=l , 2 , . . . } 


satisfy the postulates for an NHPP with mean function 


C 

1 , 


M(t) = l (1 - e U )(-dL(x)) . 
0 


Theorems 4.1 through 4.5 are generalizations of relationships 
between uniform order statistics and homogeneous Poisson processes 
(see Karlin and Taylor [13]). These relationships extend to order 
statistics from general distribution and NHPP's using the time- 
transformation which changes an HPP into an NHPP (see Cinlar [5]). 

Theorem 4.6 is somewhat more general: from Theorems 4.2 and 4.3 

(N^(t) , 0 < t < s|N^(s) = n} and {N^(t) , 0 < t < s|N^(s) = n} 

are identically distributed. Since T^ and T^ are identical and 

the occurrence of the events (T^ = s} and {T^ = s} just depends on 

the behavior of their respective processes during [0,s] , the 

conditional processes in the statement of Theorem 4.6 will be identically 
distributed. 
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Theorem 4.7 follows from various forms of the strong law of 
large numbers. Theorem 4.8 follows from various forms of multivariate 
central limit theorems. For an HPP the limit is a Brownian motion; 
for the NHPP with mean function M(t) the limiting Gaussian process 
has covariance function Cov(s,t) = M(min(s,t)> . For a process of 
uniform order statistics the limit is a Brownian bridge; for an IIDOS 
process with mean function M(t) the limiting Gaussian process has 
covariance function Cov(s,t) = M(s) (M(°°)-M(t) ) /M(°°) , s < t ; cf . 
Billingsley [2, Theorem 16.4]. For the DET/EOE process we note that 

Var(N k (t)/ ,/! T) = k _1 E VarJl [Q t] (T.(A.))^ 

_1 -At -A.t -A.t _ 

= k E ( (1 -e 1 ) (1- (1-e 1 )T + e 1 (0-(l-e 1 ) 2 ) 

i 

. -A.t -A.t 
= k _1 E e 1 (1-e 1 ) 

i 

= k 1 E exp (-L~^ (i/k) t) (1-exp (-L' 1 (i/k) t) ) 
i 

= k" 1 E exp (-L -1 (i/k) ) (1-exp (-L -1 (i/k) ) fc ) . 


This is a Riemann sum, which converges to 

L(0) 

V(t) = \ exp (-L- 1 (x)) t (1-exp (-L- 1 (x)) t )dx 

J 0 

as k -v <x> . Similarly the limiting covariance is 

L(0) 

Cov(s,t) = l exp (-L -1 (x)) S (1-exp (-L“ 1 (x)) t dx , 

J 0 
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s < t . As an example (which is tractible) consider case 2 of Section 2: 
L(A) = v exp (— A/C) ; L ^(x) = — £ log (x/v) , so the limiting covariance 


is 


f v 

0 (s,t) = I (x/v)^ S (l-(x/v)^ t )dx 

J 0 

f 1 J 

= vV u ' 

JO 


£s £t. 

■ (1-u ) du 


V g t 

(Cs+1) (£s+£t+l) 


The covariance of the IIDOS process with the same L(-) is 

OO 00 

Cov^s.t) = \ (l-e _As )dL(X) f e~ At dL(\)/L(co) 

J o Jq 


v (1 - 




Cs+r £t+i 


V S t 


(£s+l)(£t+l) 


s < t 


Note that Cov^(s,t) > CoVgCs.t) , as expected. 

Proof of Theorem 4.9 : Let X.^ , i=l s 2,... , be independent 

Exponential random variables with rates X. , i=l,2,... , respectively, 
k 1 

Define N^Ct) = E Ijq t j(xp > 0 < t , k=l,2,...; these processes are 
all defined on the same probability space and satisfy N, (t) < N. (t) , 

tC “ Kt X 

0 < t , k=l,2,..., with probability one. By monotone convergence, 

lim N, (t) exists (possibly °° ) with probability one. The limit 
k-*» 

OO 

Ijq t j(xp must be a nondegenerate process because EX^ < 00 . Thus 
we have convergence with probability one to a DET/EOS process with an 
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infinite number of events. 


Proof of Theorem 4.10 : The finite dimensional joint distributions 

of an IIDOS process with mean function M^(*) are of the form 

P{N k (ti) = n 1 , N k (t 2 ) = n 2 , ...,N k (t m > = n^} 


\ m i. M, (°°)-n 

. . . 3 (1_ w } 

X 1 1 2 ' ‘ ’ 1 m I 3=1 


where 0 = t < < t 2 < . . . < < » , F k (0 = M fe ( • ) /l^C 00 ) 


x . — n^ “ n^ ^ , J 1 , 2 j . . .m j and 


M k («>) 


i. x 0 ... x 
12 m> 


M k («>) : 


m 

V i 2* CM k ( "> “ \ V 1 


If lim M, (t) = M (t) such that M («>) < <» , then, as k -> m , the 

tC °° 00 

k-*» 

above joint distribution converges to 


M (<») \ m i. 

* . ) < f „<v - <i - F » (t » )) 

X 1 1 2 * ‘ " 1 m / 3 1 


M (oo)-n 

00 in 


the joint distribution of an IIDOS process with mean function M (•) . 

J OO 

If lim M^t) = M (t) such that M^eo) = » then, as k -*- « , the above 

k>03 

joint distribution may be rewritten as 
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but 


M k (») > 

\ u 


] 11 

• • • 1 

2 m 

i 

h 


i . 


m / ' \ m 1. 

(\(“)) . . . .VVV-VVi^^g^H) 

\ X 1 1 2 * * " 1 m / J_1 


n (Mjt ) - M ro (t )) J lim M (co) : 

j=l ^ — : — i k-*» - k 


1. , 

J '• 




-n ) : V m k c-) / 


M, (°°)-n 
k m 


lim M (°°) I 
k-*» — 




= 1 


M^ 00 ) (M k (“)-n m )! 


and 


l™ (■ \ (t m ) \ 

V " V“> / 


M k ( " ) " n m 


= expC-Mjt^)) , 


so the joint distribution converges to 

m i . 

II (M (t . ) - M (t . n J exp(-M (t )) 
00 J 00 j-l ^ °° v m' ' 

J ^ i * 


i . : 
J 


^ |exp(-(M oo (t j )-M aj (t j _ 1 ))) ( M op (t j ) - J j 

1 j ' ' 


the joint distribution of an NHPP with mean function M (•) , cf. 

oo v ' 5 

Miller [18]. 


Proof of Theorem 4.11 : It can be shown as above that the finite 

dimensional joint distributions of a sequence of NHPP's with mean functions 
M^(*) converge to those of an NHPP with mean function as k ->- 00 . 
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Proof of Theorem 6.1: Since L(X) is integer-valued, 

0° — y 

l L(X)dX < 00 if and only if lim L(X) = 0 and f L(X) dX < 00 . 

•'0 X - * 00 J 0 

But f L(A)dA = E \ ± , where X ± = L _1 (i) . 

JO i=l 

Proof of Theorem 6.2: Since is the largest rate, 

n 

P(EA. <00 )-1 if and only if P(A^< °°) = 1 ; < X) = 

i=l 1 

( (L(0) - L(A))/L(0)) n , so P(A ± < ») = 1 if and only if lim L(A) = 0 


Proof of Theorem 6.3 : The largest rate is finite if and only 

if lim L(A) = 0 (if Y ^ Exponential (1), then = L _1 (Y 1 > < ») • 

A-s-co _ 

By Theorem 6.1 the conditions lim L(A) = 0 and 

>V » 

( L(A)dA < ", 0 < u < 00 , are equivalent to £ A^ < “ > where 

J 0 ’ i=1 

* 

X. = L” 1 (i) , i=l, 2, . . . . Thus it suffices to show that P(z A. < °°) “ 1 > 

Ju 00 

where Z ' is over all A. < h , if and only if E A^ < “ • 


Let N. = E 1/, . n (A.) , i=l,2,... , these are i.i.d. Poisson 

1 j=l U i=l, A i J J 

•k 00 

random variables with mean 1; then E A . < E N A anc * 

1 i=l 

, OO oo 

E(E A.) < E( E N.A.) = E EN.A, = E A. . Therefore EA. < » implies 
1 ” i=l 1 1 i=l i=l 

EC^'a.) < 00 j which in turn implies Ptl^'A^ < «) =1 • Conversely, 

OO 

suppose that E A . = 00 • We shall randomly select a subset S of A : 


If N. > 1 , A. . belongs to S , if N ± = 0 , A ± . 


does not . 


Thus each A , i-1,2,..., is randomly and independently selected with 
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probability 1-e 1 > 1/2 ; thus E A. . = also. Note, however, 

S 

that for each 1 e S there exists a A.. such that X. 11 <A. / .. < X. . 

l+l J(i; i+l J fi; - i 

Therefore E X. , < E A.,.. and we have a subset of A whose sum 
s i+l j(i) a, 

diverges with probability one. 


Proof of Theorem 6.4: There exists y , 0 < y < °° , such that 


- Xe- U 

Jo 


dL(X) < - l XdL(X) -\ dL(X) = - l XdL(X) + L(y) . 


Integration by parts gives - I XdL(X) = I L(X)dX - yL(y) + lim XL(X) . 

Jo Jo X-K) 

- 

But yL(y) < \ L(X)dX , which in turn is finite by the admissibility 


assumption. Thus 


XdL(X) < 00 , giving m(t) < 00 . Similarly, 


M(t) < 


-s: 


XdL(X) - l dL(X) = 


-fi 


AdL(A) + L (1) , which is finite 


by admissibility of L(-) . Conversely, suppose M(t) = 


(1-e ^ t ) dL(A ) < 00 a nd lim M(t) = 0 . But 1-e ^ t > 1/2 


for log2/t < A < oo , which implies M( 
1/2 L(log2/t) . 


00 

t) > - f 

J(log2)/t 


1/2 dL(X) = 


Thus lim M(t) = 0 implies lim L(X) = 0 and we have admissibility. 
t-K) X-h» 

Proof of Theorem 6.8 : M^(*) is clearly an admissible mean function 

for an IIDOS/EOS model. If L(*) is the complementary mean function of 
the NHPP A t then the complementary mean function for the IIDOS A is 

L]_(*) = <[M(«) + 1/2] /M(a) ) L( • ) = ([L(0) + l/2]/L(0)) L(-) 
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and L^A) = L(X) + cL(X)/L(0) , |c| < 1/2 , 

OO 

|m(t) - m 1 (t) | = | J Ae _At (dL(A) - d^CX)) | 

r 00 

= 1 c f f Xe -At dL(X)/L(0) 

JO 


- 2 sup Ae At = 2 t e _1 . 

X 

Proof of Theorem 6.9: Lq(A) = [L(A) + 1/21 so |l(X) - L 0 (X) I < 1/2 

Two extreme cases are L^A) = [L(X) + 1] and L £ (A) = [L(X)] , 

which satisfy 


L u (X) > L(A) > L £ (A) 

and 

L u (X) > L q (X) > L £ (A) . 

Since these are step functions and 1—e ^ is increasing in A , 

M u (t) = - J(l-e _Xt )dL u (A) > M(t) > - J*(l-e _At )dL £ (A) = M £ (t) 

and 

M u (t) > M 0 (t) > M £ (t) . 

But 

M u (t) - M £ (t) = (l-e _A " ° 5 ) - Cl-e" A, °) = l ; 

therefore | M( t) - M Q (t)| < 1 , 0 < t < «, . Similarly, using the 

fact that Ae At is unimodal in \ with mode 1/t , one can bound it 
by the sura of two monotone functions: 

fpA) = 6 1 [0,l/t] (A) + t ±e 1 1 [l/t,«) (A) 
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and 


f 2 .(*) = 


t' 1 e' 1 


1 [0,l/t] (A) 


. -At - 

Ae 1 


[l/t,“) 


(A) 


and use a combination of the above analyses to get 

|m(t) - m n (t) | < sup f, (A) + sup f„(A) = 2(te) 1 . 

U A A 

At the end of Section 6 the Geometric Order Statistic model is 
mentioned. This model might be preferred to an EOS model when the time 
parameter is integer-valued and corresponds to successive independent 
uses of the program. In this case the times until failure have a 
geometric distribution. 


P(X = t) = (1 - p) fc 1 p , 
P(X > t) = (1 - p) 1 , 


t = 1,2,3,... ; 0 < p < 1 , where p is the failure probability on a 

given execution of the program. Denote the set of failure probabilities 

by P = { Pi > P 9 > Po * • * } and let this be a realization of a stochastic 
r\j 1 z j 

point process with complementary mean function L( # ) , i.e., L(p) 
equals the number of bugs in the program with failure probabilities equal 
to or greater than p . Let , X 2 , ... be independent Geometric 

random variables with parameters p^ , p 2 , respectively. The 

failure process is 


N(t) = Z l [0jt] (X.) , t=l,2,3, . . . . 
For deterministic p , the mean function is 

CO 

M(t) = EN(t) = 2 P(X. < t) 

i=l X 


= E (l-U-p ) C ) , t=l,2, . . . , 
i=l 


and its intensity is 
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m(t) = M(t) - M(t-l) 


= Z (l-p,)* 1 p, , t=l , 2, . . . . 
i=l 1 1 


In general 


M(t) = 


(l-d-p) 11 ) dL(p) , 


--£ 

r 1 t i - r 1 

(t) = - \ (l-p)* 1 p dL(p) = - (l-p) fc pdL(p) 

JO JO 1-p 


Let us examine m(t) : First note that {(1-p) , t=l,2,3,...} 


is a completely monotone sequence. Integration with respect to a 
positve measure preserves this property; therefore {m(t) , t=l,2,3,.. 
is a completely monotone sequence. Second, consider 


Let 


m(t) = - 


f 1 

| (l-p)*" £d 

Jo 1 

£ 


pdL(p) 
-P 


exp(t log (1-p)) pdL(p) 

1 0 1-p 


= - f l I t log (1-p) A 1 pdL(p ) 

JO i=0\ i! / 1-p 

00 . . -1 

= z ( -l)V \ (-log (1-p) ) 1 (-pdL(p ) . 

i=l i! Jo 1 -p 

- ± = J (-log (1-p)) 1 pdL(p) ^ , i=0,l,2, . . . ; 


{c^/cq , i-0,1,2,...} is the moment sequence of the random variable 
-log (1-Q) where P(p < Q < p + dp) = -Cq 1 pdL(p)/(l-p) . 

If 0 < -log (1-Q) < 1 then Theorem 6.10 applies and the moment 
sequence is completely monotone. 
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But 0 < -log (1-Q) <1 is equivalent to 0 < Q < 1-e ^ ; thus a 

necessary and sufficient condition for a c.m. moment sequence is that 

the support of -pdL(p)/l-p is contained in [0,1-e , i.e., 

L(p) = 0 for p > 1-e ^ . Thus we get qualitative properties for 

Geometric Order Statistic intensity sequences that are similar to EOS 
intensity functions except that completely monotone sequences take the 
place of completely monotone functions and in Theorem 6.11 the condition 

0 < p < 1-e 1 replaces 0 < A < 1 • 
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